Skip to content

Black Forest Labs Image Models #1193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Feb 13, 2025
Merged

Black Forest Labs Image Models #1193

merged 10 commits into from
Feb 13, 2025

Conversation

saksham36
Copy link
Contributor

No description provided.

julien-c
julien-c reviewed Feb 7, 2025
View reviewed changes
@SBrandeis
Copy link
Contributor

SBrandeis commented Feb 10, 2025 •
edited
Loading

Let's add some tests!

You can find examples here:

huggingface.js/packages/inference/test/HfInference.spec.ts

Lines 776 to 861 in f85f5f7

describe.concurrent(
"Fal AI",
() => {
const client = new HfInference(env.HF_FAL_KEY);
it(`textToImage - black-forest-labs/FLUX.1-schnell`, async () => {
const res = await client.textToImage({
model: "black-forest-labs/FLUX.1-schnell",
provider: "fal-ai",
inputs:
"Extreme close-up of a single tiger eye, direct frontal view. Detailed iris and pupil. Sharp focus on eye texture and color. Natural lighting to capture authentic eye shine and depth.",
});
expect(res).toBeInstanceOf(Blob);
});
it(`automaticSpeechRecognition - openai/whisper-large-v3`, async () => {
const res = await client.automaticSpeechRecognition({
model: "openai/whisper-large-v3",
provider: "fal-ai",
data: new Blob([readTestFile("sample2.wav")], { type: "audio/x-wav" }),
});
expect(res).toMatchObject({
text: " he has grave doubts whether sir frederick leighton's work is really greek after all and can discover in it but little of rocky ithaca",
});
});
it("textToVideo - genmo/mochi-1-preview", async () => {
const res = await textToVideo({
model: "genmo/mochi-1-preview",
inputs: "A running dog",
parameters: {
seed: 176,
},
provider: "fal-ai",
accessToken: env.HF_FAL_KEY,
});
expect(res).toBeInstanceOf(Blob);
});
it("textToVideo - HunyuanVideo", async () => {
const res = await textToVideo({
model: "tencent/HunyuanVideo",
inputs: "A running dog",
parameters: {
seed: 176,
num_inference_steps: 2,
num_frames: 85,
resolution: "480p",
},
provider: "fal-ai",
accessToken: env.HF_FAL_KEY,
});
expect(res).toBeInstanceOf(Blob);
});
it("textToVideo - CogVideoX-5b", async () => {
const res = await textToVideo({
model: "THUDM/CogVideoX-5b",
inputs: "A running dog",
parameters: {
seed: 176,
num_frames: 2,
},
provider: "fal-ai",
accessToken: env.HF_FAL_KEY,
});
expect(res).toBeInstanceOf(Blob);
});
it("textToVideo - LTX-Video", async () => {
const res = await textToVideo({
model: "Lightricks/LTX-Video",
inputs: "A running dog",
parameters: {
seed: 176,
num_inference_steps: 2,
},
provider: "fal-ai",
accessToken: env.HF_FAL_KEY,
});
expect(res).toBeInstanceOf(Blob);
});
},
TIMEOUT
);

You will need to override the hf model id -> black forest labs id mapping similarily to what is done here:

huggingface.js/packages/inference/test/HfInference.spec.ts

Lines 1072 to 1076 in f85f5f7

HARDCODED_MODEL_ID_MAPPING.nebius = {
"meta-llama/Llama-3.1-8B-Instruct": "meta-llama/Meta-Llama-3.1-8B-Instruct",
"meta-llama/Llama-3.1-70B-Instruct": "meta-llama/Meta-Llama-3.1-70B-Instruct",
"black-forest-labs/FLUX.1-schnell": "black-forest-labs/flux-schnell",
};

^ keys of this dict must match the name of a model on the HF Hub

Once you've added tests, you can update the VCR tapes (= cached API responses for offline testing) by running this command:

VCR_MODE=cache pnpm run test

@SBrandeis SBrandeis self-assigned this Feb 10, 2025
@julien-c
Copy link
Member

@saksham36 let us know if any help is needed!

@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@SBrandeis SBrandeis requested a review from julien-c February 12, 2025 15:24
julien-c
julien-c reviewed Feb 13, 2025
View reviewed changes
packages/inference/src/lib/makeRequestOptions.ts
Comment on lines +89 to +95
if (provider === "fal-ai" && authMethod === "provider-key") {
headers["Authorization"] = `Key ${accessToken}`;
} else if (provider === "black-forest-labs" && authMethod === "provider-key") {
headers["X-Key"] = accessToken;
} else {
headers["Authorization"] = `Bearer ${accessToken}`;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

julien-c
julien-c approved these changes Feb 13, 2025
View reviewed changes
julien-c
julien-c approved these changes Feb 13, 2025
View reviewed changes
Copy link
Member

@julien-c julien-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @saksham36 for starting the PR and thanks @SBrandeis for pushing it over the finish line! let's go!!

@SBrandeis SBrandeis merged commit 62e314a into huggingface:main Feb 13, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@julien-c julien-c julien-c approved these changes

@hanouticelina hanouticelina Awaiting requested review from hanouticelina hanouticelina is a code owner

@SBrandeis SBrandeis Awaiting requested review from SBrandeis SBrandeis is a code owner

@coyotte508 coyotte508 Awaiting requested review from coyotte508

Assignees

@SBrandeis SBrandeis

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@saksham36 @SBrandeis @julien-c @HuggingFaceDocBuilderDev